Virtualized architecture for system parameter identification and network component configuration with reinforcement learning

ABSTRACT

One or more computing devices, systems, and/or methods for system parameter identification and network component configuration are provided. A state comprising a system parameter combination, a traffic model, and a channel assignment may be generated. A network traffic scenario is executed through a virtualized testbed using the state. A reward for the system parameter combination may be generated based upon key performance indicators output by the network traffic scenario. A reward policy and rewards generated for system parameter combinations are used to select a system parameter combination that is used to configure a network component of a communication network.

RELATED APPLICATION

This application claims priority to and is a continuation of U.S.application Ser. No. 17/349,455, filed on Jun. 16, 2021, entitled“VIRTUALIZED ARCHITECTURE FOR SYSTEM PARAMETER IDENTIFICATION ANDNETWORK COMPONENT CONFIGURATION WITH REINFORCEMENT LEARNING”, which isincorporated by reference herein in its entirety.

BACKGROUND

A communication network may comprise a variety of network componentsthat enable communication devices, such as user equipment, tocommunicate over the communication network. For example, thecommunication network may comprise base stations (e.g., a gNodeB basestation, an eNodeB base station, etc.), baseband units, core networkcomponents, antennas, repeaters, switches, radio access network (RAN)controllers, etc. Many of these network components may be configuredwith system parameters. For example, radio system parameters maycorrespond to a time a communication device can be active before beingreleased, a handover failure timer, an interval between subsequenttransmissions of radio resource control (RRC) connection requests, etc.Cell system parameters may correspond to a number of channel qualityindicator (CQI) resources available on a physical UI control channel, anominal component of a communication device transmit power, etc. Othersystem parameters may correspond to a number of attempts for a handoverto a cell better than a serving cell before the handover is attempted toa next best cell, limits on communication device transmission power in aserving cell, etc. Each of these system parameters may have numerousvalues that can be set for the system parameters leading to acombinatorial explosion problem where the number of possible behaviorsof the system increases exponentially with the number of these systemparameters.

BRIEF DESCRIPTION OF THE DRAWINGS

While the techniques presented herein may be embodied in alternativeforms, the particular embodiments illustrated in the drawings are only afew examples that are supplemental of the description provided herein.These embodiments are not to be interpreted in a limiting manner, suchas limiting the claims appended hereto.

FIG. 1 is a diagram illustrating an example scenario for systemparameter identification and network component configuration.

FIG. 2 is a flow chart illustrating an example method for systemparameter identification and network component configuration.

FIG. 3A is a diagram illustrating an example scenario for systemparameter identification and network component configuration.

FIG. 3B is a diagram illustrating an example scenario for systemparameter identification and network component configuration, where avirtualized testbed is constructed.

FIG. 3C is a diagram illustrating an example scenario for systemparameter identification and network component configuration, where avirtualized testbed executes one or more network traffic scenarios.

FIG. 3D is a diagram illustrating an example scenario for systemparameter identification and network component configuration, whererewards are assigned based upon key performance indicators.

FIG. 3E is a diagram illustrating an example scenario for systemparameter identification and network component configuration, whereselect system parameter combinations per carrier type are determined.

FIG. 3F is a diagram illustrating an example scenario for systemparameter identification and network component configuration, where anetwork component is configured with a select system parametercombination.

FIG. 4 is an illustration of a scenario featuring an examplenon-transitory machine readable medium in accordance with one or more ofthe provisions set forth herein.

FIG. 5 is an illustration of example networks that may utilize and/orimplement at least a portion of the techniques presented herein.

FIG. 6 is an illustration of a scenario involving an exampleconfiguration of a computer that may utilize and/or implement at least aportion of the techniques presented herein.

FIG. 7 is an illustration of a scenario involving an exampleconfiguration of a client that may utilize and/or implement at least aportion of the techniques presented herein.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Subject matter will now be described more fully hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific example embodiments. Thisdescription is not intended as an extensive or detailed discussion ofknown concepts. Details that are well known may have been omitted, ormay be handled in summary fashion.

The following subject matter may be embodied in a variety of differentforms, such as methods, devices, components, and/or systems.Accordingly, this subject matter is not intended to be construed aslimited to any example embodiments set forth herein. Rather, exampleembodiments are provided merely to be illustrative. Such embodimentsmay, for example, take the form of hardware, software, firmware or anycombination thereof.

The following provides a discussion of some types of computing scenariosin which the disclosed subject matter may be utilized and/orimplemented.

One or more systems and/or techniques for system parameteridentification and network component configuration are provided. Acommunication network may comprise various network components thatprovide communication capabilities for communication devices (e.g.,phones, watches, vehicles, televisions, and/or other user equipment withcommunication capabilities, such as cellular or other wirelesscommunication capabilities). These network components can be configuredwith system parameters that can affect how the network componentsoperate, how the communication devices communicate over thecommunication network, etc. The system parameters may relate to callflows, feature processing, and/or other types of parameters such astimers (e.g., 4G LTE timers), beamforming management properties, cellparameters, system radio parameters, etc.

In general, a technician may set values for the system parameters duringinstall or maintenance, or the values may be preset by vendors of thenetwork components. The use of recommended vendor values for the systemparameters of network components may not result in optimal performanceof the network components, the communication devices, and/or thecommunication network. Furthermore, the technician may merely set thevalues to either the recommended vendor values or may tweak merely a fewsystem parameters based upon prior experience. At best, the technicianmay merely change a small subset of system parameters on a try and seebasis. This is because the technician cannot test the exponentiallylarge number of system parameter combinations (e.g., hundreds ofthousands of system parameter combinations of different values could beavailable to try). This manual setting of values for the systemparameters may not result in desired performance of the networkcomponents, the communication devices, and/or the communication network.This is because there are too many system parameter and valuecombinations for a human to manually test in order to identify optimalvalues for the system parameters. For example, a single system parametermay have hundreds to thousands of values that could be set for thesystem parameter (e.g., a timer could be set from 0 seconds to 86,400seconds), and a network component may have numerous system parameters.This results in an exponentially large number of system parameter andvalue combinations that cannot be manually tested by humans.

Manually testing values for all possible system parameter combinationsof values is expensive, time consuming, and impractical due to the largescale testing that would be necessary to test all the system parametercombinations of values. Thus, there is no way to definitively configurenetwork components in a manner that provides desired performance of thenetwork components, the communication devices, and/or the communicationnetwork (e.g., optimal accessibility, retainability, IP throughput, IPlatency, cell availability, mobility, energy efficiency, etc.).Furthermore, some network functionality, such as radio access network(RAN) software, operates in the millisecond range. Such operationoutputs too much data and scripts regarding system performance whentesting system parameters, which is impractical to manually parse todetermine how network components are operating when configured with thesystem parameters.

As provided herein, a virtualized architecture is provided foridentifying system parameters, such as on a per carrier type basis,which can be used to configure network components to operate in adesired manner. The virtualized architecture implements an agent and avirtualized testbed. Communication devices (e.g., user equipment),communication traffic, baseband units, core network functionality, IPmultimedia subsystem (IMS) functionality, and/or other communicationnetwork and/or network device functionality may be emulated/simulatedthrough the virtualized testbed.

The agent is configured to generate system parameter combinations andexecute network traffic scenarios through the virtualized testbed totest the system parameter combinations. The agent generates rewards forthe system performance parameters based upon key performance indicators(KPIs) output by the network traffic scenarios. In this way, the agentmay implement reinforcement learning through the rewards and rewardpolicies to identify select system parameter combinations (e.g., on aper carrier type basis) that provide increased performance for thenetwork components, the communication devices, and/or the communicationnetwork. The key performance indicators, and thus the desiredperformance, may relate to network accessibility, retainability, IPthroughput, IP latency, cell availability, mobility, energy efficiency,etc. A select system parameter combination for a network component(e.g., a system parameter combination determined to provide moredesired/optimal performance compared to other tested system parametercombinations) may be used to configure the network component. In thisway, the virtualized architecture is used to identify select systemparameter combinations that provide improved performance compared toother system parameter combinations. The virtualized architectureidentifies the select system parameter combinations in a scalable mannerin order to test a number of system parameter combinations that wouldotherwise be impractical for humans to manually test due to the sheernumber of system parameter combinations. This architecture improves theoperational efficiency, accuracy, and/or resource consumption by networkcomponents because each network component is individually tuned withdetermined system parameter combinations tailored for each networkcomponent. In addition, this architecture enables fine grained systemparameter combination values that are targeted and personalized to eachcarrier instead of having the same system parameter values for allcarriers (in a region or nationwide).

FIG. 1 illustrates a system 100 for implementing a virtualizedarchitecture used to identify system parameters and configured networkcomponents. The virtualized architecture may comprise a virtualizedtestbed 110. The virtualized testbed 110 may be configured to emulatecarriers as a first virtual carrier 112, a second virtual carrier 114,and/or other virtualized carriers. The virtualized testbed 110 maycomprise virtualized (emulated) network functions of network componentsfrom various vendors. The virtualized testbed 110 may implement oremulate baseband control and switching functions of network components,such as radio technology components (e.g., 4G, 5G, etc.). Thevirtualized testbed 110 may implement or emulate IP transport and radiofunction synchronization with a network time protocol (NTP) server orother type of server. The virtualized testbed 110 may implement oremulate baseband capacity that supports multiple carriers, where acarrier is a cell within a particular frequency band (e.g., 3GPP 4Gcarrier can be one of the following bands: 2, 4, 5, 13, or 66). Thevirtualized testbed 110 may implement or emulate a baseband unit thatcan either be a single unit or is disaggregated to a central unit (CU)and distributed unit (DU).

The virtualized testbed 110 may emulate interfaces to emulatedcommunication devices, such as radio units and other user equipment(UE), that are emulated using a UE emulator. The UE emulator mayimplement large scale software based user equipment virtualizingtelecommunication technologies (e.g., 3GPP technologies) as emulatedcommunication devices. The UE emulator may implement various differentnetwork traffic scenarios, which may correspond to a large scale numberof UEs, UE category type mix, frequency band mix, mobility mix,different traffic types (e.g., high throughput, low latency, etc.), 3GPPfeature mix (e.g., carrier aggregation, non-carrier aggregation,multiple input multiple output), application mix (e.g., operation of avideo streaming app, a social network app, a gamming app, etc.), etc.The UE emulator may implement channel emulation enabling different radiofrequency (RF) conditions per emulated communication device, such asrural, urban, pedestrian (e.g., a smart watch traveling slowly down asidewalk), high mobility (e.g., a phone traveling along a highway at ahigh rate of speed), etc., along with reflecting channel impairmentssuch as noise, interference, and distortion.

The UE emulator may implement latency simulation in order to emulatedifferent delay and delay variation environments. The UE emulator mayimplement a common public radio interface connection, an enhanced commonpublic radio interface connection, or other type of connection to adigital unit of a radio function (e.g., of a UE) in order to carry datasuch as IQ data corresponding to user plane information in the form ofin-phase and quadrature phase modulation data (e.g., digital basebandsignals). The UE emulator may implement cloudification through softwaredefined radios (SDRs), virtual digital signal processing (DSP) (e.g.,dual tone multi frequency, codecs, etc.), and hardware acceleration(e.g., GPUs, field programmable gate arrays, etc.).

The virtualized architecture is constructed with software defined radioand network virtual network functions (VNFs), and has an agent 102configured to utilize reinforcement learning techniques to determinewhat system parameter values to select for configuring networkcomponents. The agent 102 is configured to construct a state 104comprising a system parameter combination 116 corresponding to valuesselected for a combination of system parameters of a network componentto test through a network traffic scenario (e.g., 4G LTE timers, 5Gbeamforming management parameters, etc.). The state 104 may alsocomprise traffic models (e.g., Poisson distributions of UEs in the cell)and/or channel assignments 118 (e.g., indoor or outdoor urban, etc.). Atraffic model may correspond to a carrier type, a number ofcommunication devices to emulate, and/or other traffic parameters forsimulating communication of communication devices over a communicationnetwork of network components as emulated communication devicescommunicating over a virtualized communication network of emulatednetwork components. A channel assignment may correspond to an insideenvironment, an outdoor environment, an urban environment, and/or anyother type of environment.

The agent 102 may apply the state 104 to the virtual carriers of thevirtualized testbed 110, such as by applying the system parametercombination 116 to the first virtual carrier 112, the second virtualcarrier 114, and/or other virtual carriers. The agent 102 may perform anaction 106 to execute a network traffic scenario through the virtualizedtestbed 110 using the traffic models and the channel assignments 118. Inthis way, communication of emulated communication devices over thevirtualized communication network and operation of emulated networkcomponents configured according the system parameter combination 116 issimulated through the virtualized testbed 110 using the traffic modelsand the channel assignments 118.

During the network traffic scenario of simulating the virtualizedcommunication network, various information that provide observability ofall events in the network, may be tracked, such as accessibility,retainability, IP throughput, IP latency, cell availability, mobility,energy efficiency, etc. An example of an accessibility KPI is a RadioResource Control (RRC) connection establishment success rate. An exampleof a retainability KPI is a dropped calls rate. An example of mobilityKPIs are the intra or inter frequency handover success rates. An exampleof availability KPI is the E-UTRAN cell availability measuring thecapability to provide the service to the users. This information may betracked as key performance indicators 120. The agent 102 receives thekey performance indicators 120 as feedback, and builds rewards 108 forsystem parameter combinations as the system parameter combinations aretested through network traffic scenarios. The agent 102 may implementreinforcement learning techniques to select a system parametercombination based upon rewards assigned to the system parametercombinations and/or based upon one or more reward policies used toweight key performance indicators in a particular manner in a rewardformula used to assign rewards. This selected system parametercombination may provide more desirable performance than other systemparameter combinations for a particular carrier type, such as betteraccessibility, retainability, IP throughput, IP latency, cellavailability, mobility, energy efficiency, etc.

An embodiment of system parameter identification and network componentconfiguration is illustrated by an exemplary method 200 of FIG. 2 ,which are further described in conjunction with system 300 of FIGS.3A-3E. A virtualized architecture may implement an agent 302 configuredto test system parameter combinations, as illustrated by FIG. 3A. Theagent 302 can perform the test in a scale out manner so that any numberof system parameter combinations can be tested in order to identify aselect system parameter combination that can be used to configure anetwork component to operate in a desired manner. For example, there maybe an exponentially large number of system parameter combinations (e.g.,hundreds of thousands of combinations of values that can be assigned toa set of system parameters of a network component). The agent 302 isconfigured to generate the system parameter combinations, test thesystem parameter combinations, and evaluate test results in order toselect a system parameter combination that results in the desiredoperation of the network component. An example of system parametercombination includes connectivity timers like the handover failure timeand cell parameters like the nominal component of the UE transmit powerfor the Physical UL Control Channel (PUCCH) and/or the nominal componentof the UE transmit power for the Physical UL Shared Channel (PUSCH). Thedesired operation may correspond to accessibility, retainability, IPthroughput, IP latency, cell availability, mobility, energy efficiency,etc.

The agent 302 may receive system parameters as input. In someembodiments, the system parameters may be specified per vendor, such aswhere the system parameters are for a particular network component modelmanufactured by the vendor. For example, a first set of systemparameters 304 for a first network component of a first vendor, a secondset of system parameters for a second network component of a secondvendor, and/or other system parameters may be received as input. Theagent 302 may generate system parameter combinations based upon thesystem parameters. For example, the agent 302 may generate a firstsystem parameter combination that specifies a first set of values forthe first set of system parameters 304 for the first network componentof the first vendor. The agent 302 may generate a second systemparameter combination that specifies a second set of values for thefirst set of system parameters 304 for the first network component ofthe first vendor. At least one value of the second set of values may bedifferent than at least one value of the first set of values. The agent302 may generate a third system parameter combination that specifies athird set of values for the first set of system parameters 304 for thefirst network component of the first vendor. At least one value of thethird set of values may be different than at least one value of thefirst set of values and at least one value of the second set of values.

In some embodiments, a system parameter combination may correspond tosystem parameters of a single network component of a single vendor. Insome embodiments, a system parameter combination may correspond tosystem parameters of multiple network components of a single vendor ormultiple vendors. In some embodiments, a system parameter combinationmay correspond to system parameters of a single network component for asingle carrier type. In some embodiments, a system parameter combinationmay correspond to system parameters of multiple network components for asingle technology, a single carrier type, multiple carriers or multipletechnologies (e.g., 4G, 5G, 6G etc.).

The agent 302 is able to scale out to generate and test a large numberof system parameter combinations that humans would not be able tomanually generate and test since there could be an exponentially largenumber of system parameter combination. In this way, the agent 302 maygenerate a plurality of system parameter combinations that can be testedby the agent 302 in order to identify select system parametercombinations per carrier type that provide desired operation of acommunication network for the various carrier types.

The agent 302 may initiate state processing 306 in order to initialize avirtualized testbed 310 for testing the plurality of system parametercombinations, as illustrated by FIG. 3B. In an embodiment of initiatingthe state processing 306, the agent 302 may initialize the carriers. Inan example, a first carrier corresponds to an emulation of a firstbaseband unit 316 of a first vendor, an emulation of a first corenetwork 318, and/or an emulation of a first IP multimedia core networksubsystem (IMS) 320. A second carrier may correspond to an emulation ofa second baseband unit of a second vendor, an emulation of a second corenetwork, and/or an emulation of a second IMS. In this way, the agent 302may initialize any number of carriers through the virtualized testbed310.

As part of initializing the virtualized testbed 310, the agent 302 mayinitialize traffic models. A traffic model may correspond to carriertypes, numbers of communication devices to emulate (e.g., a number ofUEs to emulate), and/or other traffic information. An example of atraffic model includes parameters that can be used for 3GPP 4G/5Gnetwork planning and optimization such as traffic profiles made ofsignaling data, bandwidth, busy hour session attempts, number ofsimultaneous RAB (Radio Access Bearers) bearers and number ofsimultaneous EPS (Evolved Packet Systems) bearers. As part ofinitializing the virtualized testbed 310, the agent 302 may initializechannel assignments. The channel assignments may correspond to an urbanenvironment, an indoor environment, an outdoor environment, and/or otherenvironments where network components and/or communication devices maybe communicating. As part of initializing the virtualized testbed 310,the agent 302 may specify a duration for simulating a network trafficscenario (e.g., a simulation of a month of operation, a simulation of ayear of operation, etc.). In this way, the virtualized testbed 310 maybe initialized and the plurality of system parameter combinations may begenerated for testing through the virtualized testbed 310.

A UE emulator 312 may be initialized through the virtualized testbed310. The UE emulator 312 may be configured to emulate any number ofcommunication devices as emulated communication devices 314. The UEemulator 312 may implement large scale software based user equipmentvirtualizing telecommunication technologies (e.g., 3GPP technologies) asthe emulated communication devices 314. The UE emulator 312 mayimplement various different network traffic scenarios, which maycorrespond to a large scale number of UEs, a UE category type mix, afrequency band mix, a mobility mix, different traffic types, 3GPPfeature mix, an application mix, etc. The UE emulator 312 may implementchannel emulation enabling different RF conditions per emulatedcommunication device, such as rural, urban, pedestrian, high mobility,etc., along with reflecting channel impairments such as noise,interference, and distortion. The UE emulator 312 may implement latencysimulation in order to emulate different delays and delay variationenvironments. The UE emulator 312 may implement a common public radiointerface connection, an enhance common public radio interfaceconnection, or other type of connection towards a digital unit of aradio function (e.g., the emulated communication device 314) in order tocarry data such as IQ data corresponding to user plane information inthe form of in-phase and quadrature phase modulation data. The UEemulator 312 may implement cloudification through software definedradios, virtual DSP, and hardware acceleration. That is, networkfunctions and/or UE functions may be hosted in data centers (central oredge). UE emulation is implemented through cloudification. From adeployment perspective, the UE emulator 312 may also be deployed on baremetal, but at an increase cost.

During operation 202 of method 200, a state may be generated for eachsystem parameter combination, as illustrated by FIG. 3C. The state maybe generated as part of the state processing 306 implemented by theagent 302. The state may comprise a system parameter combination 324 andtraffic model and channel assignments 322. In some embodiments, theremay be more than one traffic model and/or channel assignment, and thusmultiple states may be generated for each combination of traffic modelsand channel assignments in association with the system parametercombination 324.

During operation 204 of method 200, a network traffic scenario isexecuted through the virtualized testbed for each of the states. Thenetwork traffic scenario is executed to simulate communication of thecommunication network amongst the emulated communication devices 314 andemulated network components (e.g., operation of the first baseband unit316, the first core network 318, and the first IMS 320) according thesystem parameter combination 324 and the traffic model and channelassignments 322 for the duration of the simulation. In some embodimentsof executing the network traffic scenario, a configuration action isgenerated based upon baseband unit system parameters within the systemparameter combination 324. The configuration action is executed byapplying the baseband unit system parameters to the first baseband unit316.

In some embodiments of executing the network traffic scenario, operationof the emulated communication devices 314 is simulated through thevirtualized testbed 310, such as by the UE emulator 312. In someembodiments of executing the network traffic scenario, software definedradios, virtual digital signal processing, and/or hardware accelerationis implemented by the UE emulator 312. In some embodiments of executingthe network traffic scenario, baseband control and switching functionsof a radio technology component may be emulated through an emulatednetwork component, such as the first baseband unit 316 or other emulatednetwork component not illustrated. In some embodiments of executing thenetwork traffic scenario, latency simulation corresponding to one ormore communication delays and delay variation environments may beimplemented. For example, communication delay between an emulatedcommunication device and an emulated network component may be simulatedduring the network traffic scenario. In some embodiments of executingthe network traffic scenario, channel emulation may be implemented toenable different radio frequency propagation conditions per emulatedcommunication device 314. In some embodiments of executing the networktraffic scenario, a common public radio interface connection, anenhanced common public radio interface connection, or any otherconnection may be emulated for transmission of user plane informationbetween the emulated communication devices 314 and emulated basebandunits, such as the first emulated baseband unit 316.

In some embodiments of executing the network traffic scenario, the UEemulator 312 may run traffic cases based upon the channel assignment foremulating communication network traffic under conditions of the channelassignment. In some embodiments of executing the network trafficscenario, one or more traffic scenarios may be implemented during thenetwork traffic scenario. The one or more traffic scenarios maycorrespond to a number of emulated communication devices 314 that are tobe operational during a traffic scenario, emulated communication devicecategories (e.g., types and models of the emulated communication devices314), frequency bands for communication, mobility (e.g., communicationdevices traveling fast along a highway, communication devices travelingslowing or at a standstill in a residential neighborhood, etc.), traffictypes (e.g., high throughput, low latency, etc.), and/or applicationtypes (e.g., operation of a social media app hosted on an emulatedcommunication device).

The agent 302 may receive key performance indicators 330 as output fromone or more network traffic scenarios being executed by the virtualizedtestbed 310, as illustrated by FIG. 3D. The key performance indicators330 may be the output of simulations performed to test the systemparameter combinations under one or more combinations of traffic modelsand/or channel assignments. The key performance indicators 330 maycomprise an accessibility key performance indicator, a retainability keyperformance indicator, an IP latency key performance indicator, an IPthroughput key performance indicator, a cell availability keyperformance indicator, a mobility key performance indicator, an energyefficiency key performance indicator, and/or other types of performanceindicators.

During operation 206 of method 200, the agent 302 may input the keyperformance indicators 330 into a reward model 308 for assigning rewardsto each system parameter combination that was tested through thevirtualized testbed 310. For example, a reward for a system parametercombination may be calculated based upon key performance indicatorsoutput during a network traffic scenario that utilized the systemparameter combination. The reward may be used to update the reward model308. In some embodiments, an experience comprising the system parametercombination, a traffic model used during the network traffic scenario, achannel assignment used during the network traffic scenario, a durationof the network traffic scenario, and/or the reward may be stored as anexperience.

The experiences may be evaluated using one or more reward policies toselect a system parameter combination having a reward that satisfies theone or more reward policies. For example, the select system parametercombination may provide better performance than other system parametercombinations based upon the system parameter combination having a higherreward than other system parameter combinations. In some embodiments,system parameter combinations 340 may be selected on a per carrier typebasis based upon one or more reward policies, as illustrated by FIG. 3E.These system parameter combinations may be used to configure networkcomponents corresponding to the carrier types. For example, one or morereward policies may be used to select system parameter combinations forconfiguring network components on a per carrier type basis or traffictype. In some embodiments, a reward policy may weight each keyperformance indicator equally. In some embodiments, a reward policy mayweight one key performance indicator different than another keyperformance indicator (e.g., a user may specify that certain keyperformance indicators are more important than other key performanceindicators). In some embodiments, the one or more reward policies arepart of a reward system that uses the one or more reward policies totransform KPIs into rewards, such as scores used to rank systemparameter combinations.

During operation 208 of method 200, a select system parametercombination 352 is selected from the plurality of system parametercombinations, as illustrated by FIG. 3F. In some embodiments, the selectsystem parameter combination 352 may be selected based upon the selectsystem parameter combination 352 providing desired performance, and thusis selected for use in configuring a network component 356 (e.g., aphysical network component) of a communication network.

In some embodiments, the select system parameter combination 352comprises values for radio system parameters, cell system parameters,and/or other types of parameters. In an example, radio system parametersmay correspond to a time a communication device can be active beforebeing released, a handover failure timer, an interval between subsequenttransmissions of radio resource control (RRC) connection requests, aninterval between subsequent transmission RRC connection reestablishmentrequests, a time to wait for cell reselection or RRC connection setupafter an RRC connection reject, a radio link failure (RLF) declarationtimer, an RLF recovery timer, etc. Cell system parameters may correspondto a number of channel quality indicator (CQ) resources available on aphysical UL control channel, a number of scheduling request resourcesavailable on a physical UL control channel, a nominal component of acommunication device transmit power, etc. Other system parameters maycorrespond to a number of attempts for a handover to a cell better thana serving cell before the handover is attempted to a next best cell,limits on communication device transmission power in a serving cell, anumber of orthogonal frequency division multiplexing (OFDM) symbols usedfor physical downlink control channel (PDCCH), a 1^(st) root sequencefor random access channel (RACH) preamble generation, etc.

In some embodiments of configuring the network component 356, aconfiguration command 354 is transmitted over a network to the networkcomponent 356. The configuration command 354 may be executed toconfigure the network component 356 by setting values of systemparameters to values specified by the select system parametercombination 352 in order to modify operation of the network component356 based upon the values of the select system parameter combination352. In this way, operation of the network component 356 may be improvedso that the network component 356 operates in a desired manner, such asby improving retainability, accessibility, IP throughput, IP latency,cell availability, mobility, energy efficiency, etc. of thecommunication network.

According to some embodiments, a method is provided. The method includesfor each system parameter combination corresponding to system parametersof network components of a communication network: generating a statecomprising the system parameter combination, a traffic model, and achannel assignment; executing a network traffic scenario through avirtualized testbed using the state, wherein the virtualized testbedemulates communication over a virtualized communication network betweenemulated communication devices and emulated network components basedupon the system parameter combination, the traffic model, and thechannel assignment; and generating a reward for the system parametercombination based upon key performance indicators output by the networktraffic scenario; and configuring a network component of thecommunication network based upon a select system parameter combinationselected based upon a reward policy and rewards of the system parametercombinations.

According to some embodiments, the method includes implementing, withinthe network traffic scenario, latency simulation corresponding to one ormore communication delays and delay variation environments.

According to some embodiments, the method includes emulating, throughthe network traffic scenario, at least one of a common public radiointerface connection or an enhanced common public radio interfaceconnection to transmit user plane information between the emulatedcommunication devices and an emulated baseband unit.

According to some embodiments, the method includes implementing, withinthe network traffic scenario, software defined radios, virtual digitalsignal processing, and hardware acceleration.

According to some embodiments, the method includes implementing, withinthe network traffic scenario, channel emulation enabling different radiofrequency conditions per emulated communication device.

According to some embodiments, the method includes implementing, withinthe network traffic scenario, one or more traffic scenarioscorresponding to a number of emulated communication devices, emulatedcommunication device categories, frequency bands, mobility, traffictypes, and application types.

According to some embodiments, the method includes utilizing the rewardpolicy to select system parameter combinations for configuring networkcomponents on a per carrier type basis.

According to some embodiments, the method includes utilizing the rewardpolicy to weight each key performance indicator equally.

According to some embodiments, the method includes utilizing the rewardpolicy to weight a first key performance indicator different than asecond key performance indicator.

According to some embodiments, a system is provided. The systemcomprises a processor coupled to memory, the processor configured toexecute instructions to perform operations. The operations include foreach system parameter combination corresponding to system parameters ofnetwork components of a communication network: generating a statecomprising the system parameter combination, a traffic model, and achannel assignment; executing a network traffic scenario through avirtualized testbed using the state, wherein the virtualized testbedemulates communication over a virtualized communication network betweenemulated communication devices and emulated network components basedupon the system parameter combination, the traffic model, and thechannel assignment; and generating a reward for the system parametercombination based upon key performance indicators output by the networktraffic scenario; and transmitting a configuration command over anetwork to a network component of the communication network, wherein theconfiguration command is executed to configure the network componentbased upon a select system parameter combination selected based upon areward policy and rewards of the system parameter combinations.

According to some embodiments, the operations include initializing thenetwork traffic scenario with a scenario duration, the traffic modelspecifying carrier types and a number of communication devices toemulate, the channel assignment corresponding to an environment toemulate, and carriers corresponding to the carrier types.

According to some embodiments, the operations include generating aconfiguration action based upon baseband unit system parameters withinthe system parameter combination; and executing the configuration actionby applying the baseband unit system parameters to an emulated basebandunit.

According to some embodiments, the operations include running, by a userequipment emulator, a traffic case based upon the channel assignment.

According to some embodiments, the operations include store anexperience comprising the system parameter combination, the trafficmodel, the channel assignment, a duration of the network trafficscenario, and the reward.

According to some embodiments, a non-transitory computer-readable mediumstoring instructions that when executed facilitate performance ofoperations, is provided. The operations include for each systemparameter combination corresponding to system parameters of networkcomponents of a communication network: generating a state comprising thesystem parameter combination, a traffic model, and a channel assignment;executing a network traffic scenario through a virtualized testbed usingthe state, wherein the virtualized testbed emulates communication over avirtualized communication network between emulated communication devicesand emulated network components based upon the system parametercombination, the traffic model, and the channel assignment; andgenerating a reward for the system parameter combination based upon keyperformance indicators output by the network traffic scenario; andmodifying parameter values of a network component within thecommunication network to modify operation of the network component basedupon a select system parameter combination selected based upon a rewardpolicy and rewards of the system parameter combinations.

According to some embodiments, the operations include emulating basebandcontrol and switching functions of a radio technology component throughan emulated network component.

According to some embodiments, the operations include generating thereward based upon an accessibility key performance indicator and aretainability key performance indicator.

According to some embodiments, the operations include generating thereward based upon an IP latency key performance indicator and an IPthroughput key performance indicator.

According to some embodiments, the operations include generating thereward based upon a cell availability key performance indicator and amobility key performance indicator.

According to some embodiments, the operations include generating thereward based upon an energy efficiency key performance indicator.

FIG. 4 is an illustration of a scenario 400 involving an examplenon-transitory machine readable medium 402. The non-transitory machinereadable medium 402 may comprise processor-executable instructions 412that when executed by a processor 416 cause performance (e.g., by theprocessor 416) of at least some of the provisions herein. Thenon-transitory machine readable medium 402 may comprise a memorysemiconductor (e.g., a semiconductor utilizing static random accessmemory (SRAM), dynamic random access memory (DRAM), and/or synchronousdynamic random access memory (SDRAM) technologies), a platter of a harddisk drive, a flash memory device, or a magnetic or optical disc (suchas a compact disk (CD), a digital versatile disk (DVD), or floppy disk).The example non-transitory machine readable medium 402 storescomputer-readable data 404 that, when subjected to reading 406 by areader 410 of a device 408 (e.g., a read head of a hard disk drive, or aread operation invoked on a solid-state storage device), express theprocessor-executable instructions 412. In some embodiments, theprocessor-executable instructions 412, when executed cause performanceof operations, such as at least some of the example method 200 of FIG. 2, for example. In some embodiments, the processor-executableinstructions 412 are configured to cause implementation of a system,such as at least some of the example system 100 of FIG. 1 and/or atleast some of the example system 300 of FIGS. 3A-3E, for example.

FIG. 5 is an interaction diagram of a scenario 500 illustrating aservice 502 provided by a set of computers 504 to a set of clientdevices 510 via various types of transmission mediums. The computers 504and/or client devices 510 may be capable of transmitting, receiving,processing, and/or storing many types of signals, such as in memory asphysical memory states.

The computers 504 of the service 502 may be communicatively coupledtogether, such as for exchange of communications using a transmissionmedium 506. The transmission medium 506 may be organized according toone or more network architectures, such as computer/client,peer-to-peer, and/or mesh architectures, and/or a variety of roles, suchas administrative computers, authentication computers, security monitorcomputers, data stores for objects such as files and databases, businesslogic computers, time synchronization computers, and/or front-endcomputers providing a user-facing interface for the service 502.

Likewise, the transmission medium 506 may comprise one or moresub-networks, such as may employ different architectures, may becompliant or compatible with differing protocols and/or may interoperatewithin the transmission medium 506. Additionally, various types oftransmission medium 506 may be interconnected (e.g., a router mayprovide a link between otherwise separate and independent transmissionmedium 506).

In scenario 500 of FIG. 5 , the transmission medium 506 of the service502 is connected to a transmission medium 508 that allows the service502 to exchange data with other services 502 and/or client devices 510.The transmission medium 508 may encompass various combinations ofdevices with varying levels of distribution and exposure, such as apublic wide-area network and/or a private network (e.g., a virtualprivate network (VPN) of a distributed enterprise).

In the scenario 500 of FIG. 5 , the service 502 may be accessed via thetransmission medium 508 by a user 512 of one or more client devices 510,such as a portable media player (e.g., an electronic text reader, anaudio device, or a portable gaming, exercise, or navigation device); aportable communication device (e.g., a camera, a phone, a wearable or atext chatting device); a workstation; and/or a laptop form factorcomputer. The respective client devices 510 may communicate with theservice 502 via various communicative couplings to the transmissionmedium 508. As a first such example, one or more client devices 510 maycomprise a cellular communicator and may communicate with the service502 by connecting to the transmission medium 508 via a transmissionmedium 507 provided by a cellular provider. As a second such example,one or more client devices 510 may communicate with the service 502 byconnecting to the transmission medium 508 via a transmission medium 509provided by a location such as the user's home or workplace (e.g., aWiFi (Institute of Electrical and Electronics Engineers (IEEE) Standard502.11) network or a Bluetooth (IEEE Standard 502.15.1) personal areanetwork). In this manner, the computers 504 and the client devices 510may communicate over various types of transmission mediums.

FIG. 6 presents a schematic architecture diagram 600 of a computer 504that may utilize at least a portion of the techniques provided herein.Such a computer 504 may vary widely in configuration or capabilities,alone or in conjunction with other computers, in order to provide aservice such as the service 502.

The computer 504 may comprise one or more processors 610 that processinstructions. The one or more processors 610 may optionally include aplurality of cores; one or more coprocessors, such as a mathematicscoprocessor or an integrated graphical processing unit (GPU); and/or oneor more layers of local cache memory. The computer 504 may comprisememory 602 storing various forms of applications, such as an operatingsystem 604; one or more computer applications 606; and/or various formsof data, such as a database 608 or a file system. The computer 504 maycomprise a variety of peripheral components, such as a wired and/orwireless network adapter 614 connectible to a local area network and/orwide area network; one or more storage components 616, such as a harddisk drive, a solid-state storage device (SSD), a flash memory device,and/or a magnetic and/or optical disk reader.

The computer 504 may comprise a mainboard featuring one or morecommunication buses 612 that interconnect the processor 610, the memory602, and various peripherals, using a variety of bus technologies, suchas a variant of a serial or parallel AT Attachment (ATA) bus protocol; aUniform Serial Bus (USB) protocol; and/or Small Computer SystemInterface (SCI) bus protocol. In a multibus scenario, a communicationbus 612 may interconnect the computer 504 with at least one othercomputer. Other components that may optionally be included with thecomputer 504 (though not shown in the schematic architecture diagram 600of FIG. 6 ) include a display; a display adapter, such as a graphicalprocessing unit (GPU); input peripherals, such as a keyboard and/ormouse; and a flash memory device that may store a basic input/outputsystem (BIOS) routine that facilitates booting the computer 504 to astate of readiness.

The computer 504 may operate in various physical enclosures, such as adesktop or tower, and/or may be integrated with a display as an“all-in-one” device. The computer 504 may be mounted horizontally and/orin a cabinet or rack, and/or may simply comprise an interconnected setof components. The computer 504 may comprise a dedicated and/or sharedpower supply 618 that supplies and/or regulates power for the othercomponents. The computer 504 may provide power to and/or receive powerfrom another computer and/or other devices. The computer 504 maycomprise a shared and/or dedicated climate control unit 620 thatregulates climate properties, such as temperature, humidity, and/orairflow. Many such computers 504 may be configured and/or adapted toutilize at least a portion of the techniques presented herein.

FIG. 7 presents a schematic architecture diagram 700 of a client device510 whereupon at least a portion of the techniques presented herein maybe implemented. Such a client device 510 may vary widely inconfiguration or capabilities, in order to provide a variety offunctionality to a user such as the user 512. The client device 510 maybe provided in a variety of form factors, such as a desktop or towerworkstation; an “all-in-one” device integrated with a display 708; alaptop, tablet, convertible tablet, or palmtop device; a wearable devicemountable in a headset, eyeglass, earpiece, and/or wristwatch, and/orintegrated with an article of clothing; and/or a component of a piece offurniture, such as a tabletop, and/or of another device, such as avehicle or residence. The client device 510 may serve the user in avariety of roles, such as a workstation, kiosk, media player, gamingdevice, and/or appliance.

The client device 510 may comprise one or more processors 710 thatprocess instructions. The one or more processors 710 may optionallyinclude a plurality of cores; one or more coprocessors, such as amathematics coprocessor or an integrated graphical processing unit(GPU); and/or one or more layers of local cache memory. The clientdevice 510 may comprise memory 701 storing various forms ofapplications, such as an operating system 703; one or more userapplications 702, such as document applications, media applications,file and/or data access applications, communication applications such asweb browsers and/or email clients, utilities, and/or games; and/ordrivers for various peripherals. The client device 510 may comprise avariety of peripheral components, such as a wired and/or wirelessnetwork adapter 706 connectible to a local area network and/or wide areanetwork; one or more output components, such as a display 708 coupledwith a display adapter (optionally including a graphical processing unit(GPU)), a sound adapter coupled with a speaker, and/or a printer; inputdevices for receiving input from the user, such as a keyboard 711, amouse, a microphone, a camera, and/or a touch-sensitive component of thedisplay 708; and/or environmental sensors, such as a global positioningsystem (GPS) receiver 719 that detects the location, velocity, and/oracceleration of the client device 510, a compass, accelerometer, and/orgyroscope that detects a physical orientation of the client device 510.Other components that may optionally be included with the client device510 (though not shown in the schematic architecture diagram 700 of FIG.7 ) include one or more storage components, such as a hard disk drive, asolid-state storage device (SSD), a flash memory device, and/or amagnetic and/or optical disk reader; and/or a flash memory device thatmay store a basic input/output system (BIOS) routine that facilitatesbooting the client device 510 to a state of readiness; and a climatecontrol unit that regulates climate properties, such as temperature,humidity, and airflow.

The client device 510 may comprise a mainboard featuring one or morecommunication buses 712 that interconnect the processor 710, the memory701, and various peripherals, using a variety of bus technologies, suchas a variant of a serial or parallel AT Attachment (ATA) bus protocol;the Uniform Serial Bus (USB) protocol; and/or the Small Computer SystemInterface (SCI) bus protocol. The client device 510 may comprise adedicated and/or shared power supply 718 that supplies and/or regulatespower for other components, and/or a battery 704 that stores power foruse while the client device 510 is not connected to a power source viathe power supply 718. The client device 510 may provide power to and/orreceive power from other client devices.

As used in this application, “component,” “module,” “system”,“interface”, and/or the like are generally intended to refer to acomputer-related entity, either hardware, a combination of hardware andsoftware, software, or software in execution. For example, a componentmay be, but is not limited to being, a process running on a processor, aprocessor, an object, an executable, a thread of execution, a program,and/or a computer. By way of illustration, both an application runningon a controller and the controller can be a component. One or morecomponents may reside within a process and/or thread of execution and acomponent may be localized on one computer and/or distributed betweentwo or more computers.

Unless specified otherwise, “first,” “second,” and/or the like are notintended to imply a temporal aspect, a spatial aspect, an ordering, etc.Rather, such terms are merely used as identifiers, names, etc. forfeatures, elements, items, etc. For example, a first object and a secondobject generally correspond to object A and object B or two different ortwo identical objects or the same object.

Moreover, “example” is used herein to mean serving as an example,instance, illustration, etc., and not necessarily as advantageous. Asused herein, “or” is intended to mean an inclusive “or” rather than anexclusive “or”. In addition, “a” and “an” as used in this applicationare generally be construed to mean “one or more” unless specifiedotherwise or clear from context to be directed to a singular form. Also,at least one of A and B and/or the like generally means A or B or both Aand B. Furthermore, to the extent that “includes”, “having”, “has”,“with”, and/or variants thereof are used in either the detaileddescription or the claims, such terms are intended to be inclusive in amanner similar to the term “comprising”.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing at least some of the claims.

Furthermore, the claimed subject matter may be implemented as a method,apparatus, or article of manufacture using standard programming and/orengineering techniques to produce software, firmware, hardware, or anycombination thereof to control a computer to implement the disclosedsubject matter. The term “article of manufacture” as used herein isintended to encompass a computer program accessible from anycomputer-readable device, carrier, or media. Of course, manymodifications may be made to this configuration without departing fromthe scope or spirit of the claimed subject matter.

Various operations of embodiments are provided herein. In someembodiments, one or more of the operations described may constitutecomputer readable instructions stored on one or more computer readablemedia, which if executed by a computing device, will cause the computingdevice to perform the operations described. The order in which some orall of the operations are described should not be construed as to implythat these operations are necessarily order dependent. Alternativeordering may be implemented without departing from the scope of thedisclosure. Further, it will be understood that not all operations arenecessarily present in each embodiment provided herein. Also, it will beunderstood that not all operations are necessary in some embodiments.

Also, although the disclosure has been shown and described with respectto one or more implementations, alterations and modifications may bemade thereto and additional embodiments may be implemented based upon areading and understanding of this specification and the annexeddrawings. The disclosure includes all such modifications, alterationsand additional embodiments and is limited only by the scope of thefollowing claims. The specification and drawings are accordingly to beregarded in an illustrative rather than restrictive sense. In particularregard to the various functions performed by the above describedcomponents (e.g., elements, resources, etc.), the terms used to describesuch components are intended to correspond, unless otherwise indicated,to any component which performs the specified function of the describedcomponent (e.g., that is functionally equivalent), even though notstructurally equivalent to the disclosed structure. In addition, while aparticular feature of the disclosure may have been disclosed withrespect to only one of several implementations, such feature may becombined with one or more other features of the other implementations asmay be desired and advantageous for any given or particular application.

In the preceding specification, various example embodiments have beendescribed with reference to the accompanying drawings. It will, however,be evident that various modifications and changes may be made thereto,and additional embodiments may be implemented, without departing fromthe broader scope of the invention as set forth in the claims thatfollow. The specification and drawings are accordingly to be regarded inan illustrative rather than restrictive sense.

What is claimed is:
 1. A method comprising: generating a statecomprising a system parameter combination, a traffic model, and achannel assignment; executing, based upon the state, a network trafficscenario through a virtualized testbed emulating communication over avirtualized communication network, wherein executing the network trafficscenario comprises emulating a connection to transmit informationbetween a first emulated device and a second emulated device; generatinga reward for the system parameter combination based upon an output ofthe network traffic scenario; and configuring a network component of thecommunication network based upon the reward.
 2. The method of claim 1,wherein executing the network traffic scenario comprises: implementing,within the network traffic scenario, latency simulation corresponding toat least one of one or more communication delays, one or more delayvariation environments, or packet loss simulation.
 3. The method ofclaim 1, wherein executing the network traffic scenario comprises:emulating, through the network traffic scenario, at least one of acommon public radio interface connection or an enhanced common publicradio interface connection to transmit user plane information betweenone or more emulated communication devices and an emulated basebandunit.
 4. The method of claim 1, wherein executing the network trafficscenario comprises: implementing, within the network traffic scenario,software defined radios, virtual digital signal processing, and hardwareacceleration.
 5. The method of claim 1, wherein executing the networktraffic scenario comprises: implementing, within the network trafficscenario, channel emulation enabling different radio frequencyconditions per emulated communication device.
 6. The method of claim 1,wherein executing the network traffic scenario comprises: implementing,within the network traffic scenario, one or more traffic scenarioscorresponding to a number of emulated communication devices, emulatedcommunication device categories, frequency bands, mobility, traffictypes, and application types.
 7. The method of claim 1, comprising:utilizing a reward policy to select system parameter combinations forconfiguring network components on a per carrier type basis.
 8. Themethod of claim 1, wherein configuring the network component is basedupon a select system parameter combination selected based upon a rewardpolicy and the reward.
 9. The method of claim 1, comprising: utilizing areward policy to weight a first key performance indicator of the outputof the network traffic scenario different than a second key performanceindicator of the output of the network traffic scenario.
 10. A systemcomprising: a memory comprising instructions; and a processor coupled tothe memory, the processor configured to execute the instructions tofacilitate performance of operations comprising: generating a statecomprising a system parameter combination, a traffic model, and achannel assignment; executing, based upon the state, a network trafficscenario through a virtualized testbed emulating communication over avirtualized communication network, wherein executing the network trafficscenario comprises emulating a connection to transmit informationbetween a first emulated device and a second emulated device; generatinga reward for the system parameter combination based upon an output ofthe network traffic scenario; and transmitting a configuration commandover a network to a network component of the communication network,wherein the configuration command is based upon the reward.
 11. Thesystem of claim 10, wherein the operations comprise: initializing thenetwork traffic scenario with a scenario duration, the traffic modelspecifying carrier types and a number of communication devices toemulate, the channel assignment corresponding to an environment toemulate, and carriers corresponding to the carrier types.
 12. The systemof claim 10, wherein the operations comprise: generating a configurationaction based upon baseband unit system parameters within the systemparameter combination; and executing the configuration action byapplying the baseband unit system parameters to an emulated basebandunit.
 13. The system of claim 10, wherein the operations comprise:running, by a user equipment emulator, a traffic case based upon thechannel assignment.
 14. The system of claim 10, wherein the operationscomprise: store an experience comprising the system parametercombination, the traffic model, the channel assignment, a duration ofthe network traffic scenario, and the reward.
 15. A non-transitorycomputer-readable medium storing instructions that when executedfacilitate performance of operations comprising: generating a statecomprising a system parameter combination, a traffic model, and achannel assignment; executing, based upon the state, a network trafficscenario through a virtualized testbed emulating communication over avirtualized communication network, wherein executing the network trafficscenario comprises emulating a connection to transmit informationbetween a first emulated device and a second emulated device; generatinga reward for the system parameter combination based upon an output ofthe network traffic scenario; and modifying parameter values of anetwork component within the communication network to modify operationof the network component based upon the reward.
 16. The non-transitorycomputer-readable medium of claim 15, wherein the operations comprise:emulating baseband control and switching functions of a radio technologycomponent through an emulated network component.
 17. The non-transitorycomputer-readable medium of claim 15, wherein the operations comprise:generating the reward based upon an accessibility key performanceindicator and a retainability key performance indicator.
 18. Thenon-transitory computer-readable medium of claim 15, wherein theoperations comprise: generating the reward based upon an IP latency keyperformance indicator and an IP throughput key performance indicator.19. The non-transitory computer-readable medium of claim 15, wherein theoperations comprise: generating the reward based upon a cellavailability key performance indicator and a mobility key performanceindicator.
 20. The non-transitory computer-readable medium of claim 15,wherein the operations comprise: generating the reward based upon anenergy efficiency key performance indicator.